New multi-media techniques such as Voice over IP (VoIP) and the Internet Protocol Television (IPTV) open a whole range of new multi-media services. One type of services enables viewers at home to participate in a television broadcast wherein the participant's input, which is fed back to the broadcaster, may change or influence the program in which the participant participates.
An example of such an interactive service may be an interactive karaoke show, wherein viewers at home may participate and interact with the show at the studio location. Typically such interactive service comprises a television signal comprising e.g. the music played by an orchestra at the studio location and the subtitles for presenting the text of the song on the television screen of the participating viewer. The television signal from the studio is sent via a network to a home-located terminal, e.g. a set-top box or the like, of a participating viewer and displayed by the television set of the viewer, which is connected to the terminal. The viewer may interact with the content of the television signal from the studio by using audio-video recording equipment connected to the terminal allowing the viewer to record his or her performance and to send the user signal back to the studio. A further example of an interactive television service may be home-located viewers participating in a live television quiz such that the quiz program may change on the basis of the answers of the participating viewers.
Due to the limitations of the feedback channel (i.e. the return path through which the participant's input is fed back to the broadcaster) current viewer participation in television broadcasts is limited. One example is a television program which allows a webcam signal or a text-message of a home-located viewer to appear “on screen” in a television program. No real interaction with the program content is possible. Another example of viewer participation is a television show, which allows a viewer to participate through a telephone. A telephone line has little delay and thus allowing almost real-time interaction with the studio. However, because its limited bandwidth a telephone line is only suitable for audio signals and thus not suitable for visual interactivity.
A more advanced example of viewer participation is described in an article by Rauhala et al (“A New Feature for TV Programs: Viewer Participation Through Videoconferencing”, Lecture notes in Computer Science, Vol. 3333, 2004, pp. 298-305). The authors propose the use of a videoconferencing server for processing a video signal of a participating viewer in combination with a video mixer. The mixer mixes the signal of the participating viewer with the studio signal carrying the television program signal into a single output signal. Problems related to the quality of the signal and delays were reported.
Implementation of true viewer participation in an interactive broadcast—in the sense that the participating home-located viewers and the studio-located viewers experience audiovisual interaction with each other—introduces certain problems within the context of IPTV. One problem relates to the synchronization of streams in the interactive broadcast. It is essential that the content of the television studio signal and the content of the home-recorded signal of a participating viewer are synchronized. The synchronized content of both signals may be transmitted back to the studio where it may be used as an input signal for the interactive broadcast.
In an IPTV environment however, where the broadcast signals are transmitted as steams over one or more networks, network delays are inevitably present. During an interactive TV broadcast session packetized streams are transmitted back and forth between the participating viewer and the studio over one or more networks. These streams will be subjected to unknown delays, which are caused by various sources in the network, such as transmission delays, delays due to differences in network routes and coding- and decoding delays, etc. As a consequence, the temporal relationship between the packets in the streams of the interactive broadcast, in particular the streams sent from the broadcaster to the home-located participants and the streams sent by the home-located participant back to the broadcaster, may be disturbed thereby seriously degrading the interactive service.
A further problem relates to the quality of the television signals in the interactive broadcast. The quality is defined in the various television broadcast standards, including de high quality standards like HDTV and EDTV. As a consequence, offering high quality interactive services will require the use of high quality video mixers.
Mixing high quality broadcast streams however, is a relatively time consuming process, which may take in the order of magnitude of seconds. Such mixing delays may thus provide a significant contribution to the total delay the streams in the network are subjected to. Hence, mixing of high quality broadcast streams negatively influence the synchronization of the streams in the interactive television broadcast.